Language Modeling with Generative Adversarial Networks

Generative Adversarial Networks (GANs) have been promising in the field of image generation, however, they have been hard to train for language generation. GANs were originally designed to output differentiable values, so discrete language generation is challenging for them which causes high levels of instability in training GANs. Consequently, past work has resorted to pre-training with maximum-likelihood or training GANs without pre-training with a WGAN objective with a gradient penalty. In this study, we present a comparison of those approaches. Furthermore, we present the results of some experiments that indicate better training and convergence of Wasserstein GANs (WGANs) when a weaker regularization term is enforcing the Lipschitz constraint

Clinical Parameters Prediction for Gait Disorder Recognition

Being able to predict clinical parameters in order to diagnose gait disorders in a patient is of great value in planning treatments. It is known that \textit{decision parameters} such as cadence, step length, and walking speed are critical in the diagnosis of gait disorders in patients. This project aims to predict the decision parameters using two ways and afterwards giving advice on whether a patient needs treatment or not. In one way, we use clinically measured parameters such as Ankle Dorsiflexion, age, walking speed, step length, stride length, weight over height squared (BMI) and etc. to predict the decision parameters. In a second way, we use videos recorded from patient's walking tests in a clinic in order to extract the coordinates of the joints of the patient over time and predict the decision parameters. Finally, having the decision parameters we pre-classify gait disorder intensity of a patient and as the result make decisions on whether a patient needs treatment or not.Comment: 6 page

Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking

Zero-shot transfer learning for multi-domain dialogue state tracking can allow us to handle new domains without incurring the high cost of data acquisition. This paper proposes new zero-short transfer learning technique for dialogue state tracking where the in-domain training data are all synthesized from an abstract dialogue model and the ontology of the domain. We show that data augmentation through synthesized data can improve the accuracy of zero-shot learning for both the TRADE model and the BERT-based SUMBT model on the MultiWOZ 2.1 dataset. We show training with only synthesized in-domain data on the SUMBT model can reach about 2/3 of the accuracy obtained with the full training dataset. We improve the zero-shot learning state of the art on average across domains by 21%.Comment: 9 pages. To appear in ACL 202

HUBERT Untangles BERT to Improve Transfer across NLP Tasks

We introduce HUBERT which combines the structured-representational power of Tensor-Product Representations (TPRs) and BERT, a pre-trained bidirectional Transformer language model. We show that there is shared structure between different NLP datasets that HUBERT, but not BERT, is able to learn and leverage. We validate the effectiveness of our model on the GLUE benchmark and HANS dataset. Our experiment results show that untangling data-specific semantics from general language structure is key for better transfer among NLP tasks